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Abstract 


The goal of this article is to provide a gentle introduction to the basic def- 
initions, goals and constructions in coding theory. In particular we focus on 
the algorithmic tasks tackled by the theory. We describe some of the classical 
algebraic constructions of error-correcting codes including the Hamming code, 
the Hadamard code and the Reed Solomon code. We describe simple proofs 
of their error-correction properties. We also describe simple and efficient algo- 
rithms for decoding these codes. It is our aim that a computer scientist with 
just a basic knowledge of linear algebra and modern algebra should be able to 
understand every proof given here. We also describe some recent developments 
and some salient open problems. 


1 Introduction 


Error-correcting codes are combinatorial structures that allow for the transmis- 
sion of information over a noisy channel and the recovery of the information 
without any loss at the receiving end. Error-correcting codes come in two basic 
formats. (1) The “block error-correcting code”: Here the information is broken 
up into small pieces. Each piece contains a fixed finite amount of information. 
The encoding method is applied to each piece individually (independently). The 
resulting encoded pieces (or blocks) are sent over the noisy channel. (2) The 
“convolutional codes”: Here the information is viewed as a potentially infinite 
stream of bits and the encoding method is structured so as to handle an infinite 
stream. This survey will be restricted to the coverage of some standard block 
error-correcting codes. 

Formally a block error-correcting code may be specified by an encoding func- 
tion C’. The input to Cis a message m, which is a k-letter string over some al- 
phabet S’ (typically Y' = {0,1} but we will cover more general codes as well). F 


maps m into a longer n-letter string over the same alphabet!. The mapped string 
is referred to as a codeword. The basic idea is that in order to send the message 
m over to the receiver, we transmit instead the codeword C'(m). By the time this 
message reaches the destination it will be corrupted, i.e., a few letters in C(m) 
would have changed. Say the received word is R. Hopefully R will still be able to 
convey the original message m even if it is not identically equal to C'(m). The only 
way to preserve this form of redundancy is by ensuring that no two codewords are 
too “close” to each other. This brings us to the important notion of “close” ness 
used, namely the Hamming distance. The Hamming distance between two strings 
x,y € &”, denoted A(x, y), is the number of letters where « and y differ. No- 
tice that A forms a metric, i.e., A(x,y) = 0 > 4 = y, A(z,y) = Aly, x) and 
A(x,y) + A(w,z) > A(a,z). A basic parameter associated with a code is its 
distance 1.e., the maximum value d such that any two codewords are a Hamming 
distance of at least d apart. Given a code of distance d and a received word R that 
differs from C'(m) in at most e < d—1 places, the error in the transmission can be 
detected. Specifically, we can tell that some letter(s) has been corrupted in the 
transmission, even though we may not know which letters are corrupted. In order 
to to actually correct errors we have to be able to recover m uniquely based on R 
and a bound ¢ on the number of errors that may have occurred. To get the latter 
property t has to be somewhat smaller than d—1. Specifically if ¢ < |(d—1)/2|, 
then we notice that indeed there can be at most one message m such that 
A(C(m), R) < t. (If mz and mz both satisfy A(C(m1), R), A(C(me), R) < Ft, 
then A(C(m1), C(ma2)) < A(m, R) + A(R, me) < 2t < d—1, contradicting the 
distance of C’.) Thus in an information theoretic sense R maintains the informa- 
tion contained in m. Recovering the information m efficiently from C' is another 
matter and we will come back to this topic presently. 

To summarize the discussion above we adopt the following terse notation that 
is standard in coding theory. A code C'is an [n, k, d]q code if C : vk» 3” where 
|| = q with min, yeyx {A(C(#), C(y))} = d. With some abuse of notation we 
will use C' to denote the image of the map C (i.e., C may denote the collection of 
codewords rather than the map). C is called a e-error-detecting code for e = d—1 
and a ¢t-error correcting code for ¢ = |(d— 1)/2]. 

In the remaining sections of this article we will describe some common con- 
structions of [n,k,d], for various choices of the parameters n,k,d and g. We 
will also describe the algorithmic issues motivated by these combinatorial ob- 
jects and try to provide some solutions (and summarize the open problems). 
(We assume some familiarity with algebra of finite fields [10, 19].) Before going 
on to these issues, we once again stress the importance of the theory of error- 
correcting codes and its relevance to computer science. The obvious applications 
of error-correcting codes are to areas where dealing with error becomes important 
such as storage of information on disks, CDs, and communication over modems 


' The assumption that the message is a k-letter string over Y) is just made for no- 
tational convenience. As it will become obvious, the representation of the message 
space is irrelevant to the communication channel. The representation of the encoded 
string is however very relevant! 


etc. Additionally, and this is where they become important to the theoretical 
computer scientist, error-correcting codes come into play in several ways in com- 
plexity theory — for example, in fault-tolerant computing, in cryptography, in 
the derandomization of randomized algorithms and in the construction of prob- 
abilistically checkable proofs. In several of these cases it is not so much the final 
results as the notions, methods and ingredients from coding theory that help. All 
of this makes it important that a theoretical computer scientist be comfortable 
with the methods of this field — and this is the goal of this article. A reader 
interested in further details may try one of the more classical texts [2, 11, 17]. 
Also, the article of Vardy [18] is highly recommended for a more detailed account 
of progress in coding theory. The article is also rich with pointers to topics of 
current interest. 


2 Linear Codes 


While all questions relating to coding theory can be stated in general, we will 
focus in our article on a subset of codes called linear codes. These codes are ob- 
tained by restricting the underlying alphabet ¥ to be a finite field of cardinality 
q with binary operations “+” and “.”. Thus a string in ©” can be thought of as 
a vector in n-dimensional space, with induced operations “+” (vector addition), 
and “.” (scalar multiplication). Thus a code €C C 5” is now a subset of the 
vectors. If this subset of vectors forms a “subspace” then the code is linear, as 
made formal below: 


Definition1. C C &” is a linear code if Va € XY, t,yeEC,et+y,a- ree. 


Many of the parameters of error-correcting codes become very clean in the 
case of linear codes. For instance, how does one specify a code C € X”? For 
general codes, succinct representations may not exist! However, for every linear 
code a succinct representation, of size polynomial in n does exist! In particular, 
we have the following two representations: 


1. For every [n,k,d]q linear code C there exists an n x k “generator” matrix 
G = Ge with entries from ©’ such that C = {Gzr|x € D*}. 

2. For every [n,k,d], code C there exists an (n — k) x n parity check matrix 
H = He over S' such that C = {y € ©” s.t. Hy = O}. 


Conversely, the following hold: Every n x & matrix G over »' defines an 
[n, k’, d]q code, for some d > 1 and k’ < k, Cg having as codewords {Gz|x € S*}. 
Similarly every (n—k) x n matrix H defines an [n, k’, d] code Ci,, for some d > 1 
and k’ < k, having as codewords {y € £”|Hy = 0}. 

Exercise: 


1. Prove properties (1) and (2) above. 

2. Given the generator matrix Ge of a code C, give a polynomial time algorithm 
to compute a parity check matrix He for C. 

3. Show that if G is of full column rank (# is of full row rank) then the code 
Ca (CH) is an [n,k, d], code. 


3 Some common constructions of codes 


In this section we describe some common construction of codes. But first let us 
establish the goal for this section. In general we would like to find families of 
[n, k, d], codes for infinitely many triples (n, k, d) for some fixed g. The property 
we would really like is that k/n and d/n are bounded away from zero as n > 
oo. Such a code is termed asymptotically good and the two properties k/n > 
0 and d/n > 0 are termed constant message-rate and constant distance-rate 
respectively. Unfortunately we will not be able to get to this goal in this article. 
But we will settle for what we term weakly good codes. These are codes with 
polynomial message-rate, i.e., k = Q(n*) for some € > 0 and constant distance- 
rate. 


3.1 Hamming code 


Hamming codes are defined for every positive n such that there exists an integer 
I such that n = 2'—1. Then the Hamming code of block size n over the alphabet 
{0,1} is given by an / x n parity check matrix H™“¢ whose columns are all the 
distinct I-dimensional non-zero vectors. Notice that there are exactly 2! — 1 of 
these. 


Lemma 2. For every positive integer n such that n = 2'— 1 for some integer 1, 
the Hamming code of block size n is an [n,n —1, 3]o code. 


Proof Sketch. Notice that the rank of H™™® is 1. In particular the column vectors 
containing exactly one | are linearly independent and there are / of them. Thus 
we find that the Hamming code is an [n,k, d]z code fork =n —l. 

We now move to showing that the distance of the Hamming code is 3. Notice 
that the code has no elements of weights since this would imply that two vectors 
in the parity check matrix are identical. This implies the distance is at least 
3. Now consider any two column vectors v,; and v2 in H™™¢, Notice that the 
vector vy; + v9 is also a column vector of H#®™® and is distinct from v, and 
v2. Now consider the n dimensional vector which is zero everywhere except in 
the coordinates corresponding to the vectors v1, vg and v, + ve. This vector has 
weight 3 and is easily seen to be an element of the Hamming code. Thus the 
distance of the Hamming code is exactly 3. 


The Hamming code is a simple code with a very good rate. Unfortunately 
it can only correct 1 error, definitely far from our goal of constant error-rate. 
Next we move on to a code with good error-correcting properties, but with very 
low-rate. 


3.2 Hadamard code 


A Hadamard matrix is an n x n matrix M with entries from +1 such that 
MM? = n-I, where I, is the n x n identity matrix. A Hadamard matrix 


immediately leads to an error correcting code where the rows of M are the 
codewords. This leads to a codeword over the alphabet © = {+1,—1}. We prove 
the distance property of the code first. 


Lemma 3. If M is a Hadamard matrix then any two rows agree is exactly n/2 
places. 


Proof. Say the rows of interest are the ith and jth rows. Then consider the 
element (MM*);;. This element is the sum of n terms, with the kth term being 
mizm;,. Notice that this term evaluates to +1 if my, = mj, and —1 otherwise. 
Thus if the ith and jth rows disagree in ¢ places, then (MM7),;; = (n—1t) +#. 
Since (MM*);; = 0, we have that n—2t = 0 and hence the two rows (dis)agree 
in exactly n/2 places. 


Thus the task of constructing a Hadamard code reduces to the task of con- 
structing Hadamard matrices. Constructions of Hadamard matrices have been 
a subject of much interest in combinatorics. It is clear (from Lemma 3) that for 
an n x n Hadamard matrix to exists n must be even. The converse is not known 
to be true and is still an open question. What is known is that an n x n matrix 
exists for every n of the form p—1 where pis a prime. It is also known that if an 
n, x n, Hadamard matrix exists and an n2 x ng Hadamard matrix exists, then 
an NyN2g XN Ny Matrix exists. Many other such constructions are also known but 
not all possibilities are covered yet. Here we give the basic construction which 
applies when n is a power of 2. These constructions are described recursively as 
follows: 
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Lemma4. For every |, the rows of ME™ form a [2',1,2'~"]s code. 
Proof. Left as an exercise to the reader. 


The Hadamard codes maintain a constant distance-rate. However their message- 
rate approaches zero very quickly. Next we describe a code with constant message- 
rate and distance-rate. The catch is that the code uses an alphabet of growing 
size. 


3.3. Reed Solomon code 


The Reed Solomon codes are a family of codes defined over an alphabet of 
growing size, with n < g. The more common definition of this code is not (we feel) 
as intuitive or as useful as the “folklore” definition. We present both definitions 
here, starting with the more useful one and then show the equivalence of the 
two. 


Definition5 (Reed Solomon codes). Let © be a field of size g, n < q and let 
£o,..-,;€n—1 be some fixed enumeration of n of the elements of ’. (It is standard 
to pick n = q—1 and 2; = a’ for some primitive element a?. Then for every 


k <n, the Reed Solomon code CRs.n.k.q is defined as follows: A message m = 


Mo ...Mp—1 corresponds to the degree k — 1 polynomial M(x) = yuo 7" mx'. 


The encoding of m, is Crs.nk q(t) = ¢€p...Cn—1 where c; = M(a;). 


The distance properties of the Reed Solomon codes follow immediately from 
the fact that a degree & — 1 polynomial may only have k — | zeroes unless all of 
its coefficients are zero. 


Lemma 6. For every n < q and k <n, the Reed Solomon code Cag ,, , forms 
an [n, k,n —k], linear code. 


Proof. The fact that the code is linear follows from the fact that if Mo(«) and 
M,(«) are polynomials of degree at most & — 1 then so is Mo(a) + Mi (a). The 
distance follows from the fact that if Mo(z;) = Mi (#;) for k values of j then 
Mo = M, (or equivalently if Mo(x;) — Mi(x;) is zero for k values of j, then 
Mp — M, is the zero polynomial). 


Finally for the sake of completeness we present a second definition of Reed 
Solomon codes. This definition is more commonly seen in the texts, but we feel 
this part may be safely skipped at first reading. 


Definition7 (Reed Solomon codes). Let 5’ be a field of size g with primitive 
element a, and letn=q—-—1,k <n. Let Py g(x t) be the polynomial (# — a)(a# — 
a?)---(2 — a"-*). The Reed Solomon code C2. in,k.q 18 defined as follows: A 
message m= mq...™M,_1 corresponds to the degree k — 1 polynomial M(x) = 
yoy m 2". The encoding of m, is CRs nk gl m) = ¢€o...€n—1 where c; is the 
coefficient of 7 in the polynomial Py q(z)M (a). 


Viewed this way it is hard to see the correspondence between the two defini- 
tions (or the distance property). We prove an equivalence next. 


Lemma 8. The definitions of Reed Solomon codes given in Definitions 5 and 7 
coincide for n = q — 1 and the standard enumeration of the elements of GF(q). 


Proof. Notice that it suffices to prove that every codeword according to the first 
definition is a codeword according to the second definition. The fact that the 
sets are of the same size implies that they are identical. 

Consider the encoding of m = mo...mgz_ 1. This encoding is Chs nk = 
Co..-€n—1 With ¢c; = YS iz0 mj; (at)s, To show that this is a codeword according 


to the second definition we need to verify that the polynomial C(«) = an cya" 


? w is a primitive element of the field GF(g) if a’ 4 1 for any 7 <q—1. 


has (x — a!) as a factor for every | € {1,...,n —k}. Equivalently it suffices to 
verify that C(a!) = 0, which we do next: 


where y;; = a/+!. Notice that for every j,ls.t. j +l 4 q-1, y1 F 1. 
Notice further that for every such y;; the summation an Vi = 0°. Since 
j€{0,...,k—1}, we find that y;; # 1 for every! € {1,...,¢-—1—k}. Thus for 
every 1 € {1,...,n—k}, we find that C(a!') = 0. This concludes the proof. 


3.4 Multivariate polynomial codes 


The next family of codes we describe are not very commonly used in coding 
theory, but have turned out to be fairly useful in complexity theory and in 
particular in the results on probabilistically checkable proofs. Surprisingly these 
codes turn out to be a common generalization of Hadamard codes and Reed 
Solomon codes! 


Definition 9 (Multivariate polynomial code). For integer parameters m,l 
and q with | < q, the multivariate polynomial code Cpoty,m,qg has as mes- 
sage a string of coefficients m = {Mi,,iz,...,im} With tj > 0 and D),tj; < L. 
This sequence is interpreted as the m-variate polynomial M(#1,...,2m) = 


ityeeiy Mis ijy eyo -z'm The encoding of mis the string of letters {M(a1,...,%m)} 


with one letter for every (@1,...,@%m) € Y™. 


Obviously the multivariate polynomial codes form a generalization of the 
Reed Solomon codes (again using the first definition given here of Reed Solomon 
codes). The distance property of the multivariate polynomial codes follow also 
from the distance property of multivariate polynomials (cf. [5, 13, 21]). 


Lemma 10. For integers m,l and q with < q, the code Croy,mjq 1s an [n, k, d]g 
code witthn=q",k= Gan) and d= (q—1)q""". 


° This identity is obtained as follows: Recall that Fermat’s little theorem asserts that 
vi! _ 1 = 0 for every non-zero ¥ in GF(q). Factoring the left hand side, we find 
that either y —1=0or ye y' = 0. Since y #1, the latter must be the case. 


Proof. The bound on n is immediate. The fact that the number of coefficients 
1, ..+5%m Sut. oF a; </is at mt) is a well-known exercise in counting. Finally 
the bound on the distance follows from the fact a degree / polynomial can only 
be zero for //q fraction of its inputs. (This is an easy inductive argument based 
on the number of variables. The base case is well known and inductively one 
picks a random assignment to the variables x1,...,¢%m—1 and argues that the 
resulting polynomial in #,, 1s non-zero with high probability. Finally one uses 
the base case again to conclude that the final polynomial in z,, is left non-zero 
by a random assignment to 2m.) 


It is easy to see that the code CRs. ak. is the same as the code Cpowy1,k-1,9: 
Also notice that the code Cpory,m,1,2 forms an [2™,m,2™-"]> code, same as 
parameters of the Hadamard code given by the rows of ME. It turns out that 
these two codes are in fact identical. The proof is left as an exercise to the reader. 


3.5 Concatenated codes 


Each code in the collection of codes we have accumulated above has some flaw 
or the other. The Hamming codes don’t correct too many errors, the Hadamard 
codes are too low-rate, and the Reed Solomon codes depend on a very large 
alphabet. Yet it turns out it is possible to put some of these codes together 
and obtain a code with reasonably good behavior (“polynomially good”). This 
is made possible by a simple idea called “concatenation” , defined next. 


Definition 11 (Concatenation of codes). Let C; be an [n1, k1, dig, code over 
the alphabet X) and let Cy be an [no, ko, dejg, code over the alphabet 5. If 
a qs? then the code C, © Cy is defined as follows: Associate every letter in 
57, with a codeword of Cy. Encode every message first using the code C, and 
then encode every letter in the encoded string using the code C2. More for- 
mally, given a message m € oy = Spike let Ci(m) = c1...en, € LY'. The 
encoding C; 0 €g(m) is given by c11...Cin,C21---€nyny © Y5'"”, where for every 
iE {l,..., m1}, Ci ...¢ing = Co(e;). 


Almost immediately we get the following property of concatenation. 


Lemma 12. [f C, is an [n1,k1, dig, code and if Cy is an [no, ko, del, code with 
1 = qh’, then C1 oC, ts an [ni no, ki ke, d'|g, code, for some d' > didy. 


Proof. The block size and message size bounds follow from the definition. To 
see the distance property, consider two messages m!,m? € oe, For 1 € {1,2}, 
let | .. ch be the encoding of m! using C; and let c4,.. Chins be its encoding 
using C,0C,. Notice that there must exist at least d, values of i such that c} 4 c? 
(by the distance of Cy). For every such é, there must exist at least dy values of 7 
such that. Ci; # C3; (by the distance of C2). Thus we find that C, 0 C2(m') and 


C1 0 C2(m?*) differ in at least did2 places. 


To best see the power of concatenation, consider the following simple ap- 
plication: Let C, be a Reed Solomon code with gq = 2”, n = q and k = An. 
Le., Cy is an [n,.4n, .6n]om code with n = 2™. Let Cy be the Hadamard code 
[2™,m,2™-1]2. The concatenation C, oC, is an [n?, .4nlogn, .3n7]2 code. Le., the 
resulting code has constant distance-rate, polynomial rate and is over the binary 
alphabet! Thus this satisfies our weaker goal of obtaining a weakly-good code. 
Even the goal of obtaining an asymptotically good code is close now. In particu- 
lar, the code of Justesen is obtained by an idea similar to that of concatenation. 
Unfortunately we shall not be able to cover this material in this article. 


4 Algorithmic tasks 


We now move on to the algorithmic tasks of interests: The obvious first candidate 
is encoding. 


Problem 13 (Encoding). 
INPUT: n x k matrix G and message m € D*. 
OuTPUT: C(m), where C = Cg is the code with G as the generator matrix. 


It is clear that the problem as specified above is easily solved in time O(nk) 
and hence in time polynomial in n. For specific linear codes such as the Reed 
Solomon codes it is possible to encode the codes faster, in time O(n log® n) for 
some constant c. However till recently no asymptotically good code was known 
to be encodable in linear time. In a recent breakthrough. Spielman [15] presented 
the first known code that is encodable in linear time. We will discuss this more 
in a little bit. 

The next obvious candidate problem is the decoding problem. Once again 
it is clear that if the received word has no errors, then this problem is only as 
hard as solving a linear system and thus can be easily solved in polynomial time. 
So our attention moves to the case where the received word has errors. We first 
define the error detection problem. 


Problem 14 (Error detection). 
INPUT: nx generator matrix G for a code C = Cg; and a received word RE X”. 
Output: Is R a codeword? 


The error detection problem is also easy to solve in polynomial time. We find 
the parity check matrix H for the code C and then check if HR =0. We now 
move to the problem of decoding in the presence of errors. This problem comes 
in several variants. We start with the simple definition first: 


Problem 15 (Mazrimum likelihood decoding). 

INPUT: nx generator matrix G for a code C = Cg; and a received word RE X”. 
OuTputT: Find a codeword x € C, that is nearest to R in Hamming distance. 
(Ties may be broken arbitrarily.) 


There are two obvious strategies for solving the maximum likelihood decoding 


problem: 

Brute Force 1: Enumerate all the codewords and find the one that is closest 
to R. 

Brute Force 2: For t = 0,1,...,, do: Enumerate all possible words within a 


Hamming distance of ¢ from R and check if the word is a codeword. Output the 
first match. 

Despite the naivete of the search strategies above, there are some simple cases 
where these strategies work in polynomial time. For instance, the first strategy 
above does work in polynomial time for Hadamard codes. The second strategy 
above works in polynomial time for Hamming codes (why?). However, both 
strategies start taking exponential time once the number of codewords becomes 
large, while distance also remains large. In particular, for “asymptotically good” 
or even “weakly good” codes, both strategies above run in exponential time. 
One may wonder if this exponential time behavior is inherent to the decoding 
problem. In perhaps the first “complexity” result in coding theory, Berlekamp, 
McEliece and van Tilborg [4] present the answer to this question. 


Theorem 16 [4]. The Maximum likelihood decoding problem for general linear 
codes is NP-hard. 


There are two potential ways to attempt to circumvent this result. One 
method is to define and solve the maximum likelihood decoding problem for 
specific linear codes. We will come to this question momentarily. The other hope 
is that we attempt to correct only a limited number of errors. In order to do so, 
we further parameterize the maximum likelihood decoding problem as follows: 


Problem 17 (Bounded distance decoding). 

INPUT: n x & generator matrix G for a code C = Cg; a received word R € X” 
and a positive integer t. 

Output: Find any/all codewords in C within a Hamming distance of ¢ from R. 


The hardness result of [4] actually applies to the Bounded distance decoding 
problem as well. However one could hope for a result of the form: “There exists an 
€ > 0, such that for every [n, k, d], linear code C, the bounded distance decoding 
problem for C with ¢ = ed is solvable in polynomial time”. One bottleneck to 
such a general result is that we don’t know how to compute d for a generic linear 
code. This motivates the following problem: 


Problem 18 (Minimum distance). 
INPUT: n x k generator matrix G for a code C = Cg and an integer parameter d. 
Output: Is the distance of C at least d? 


This problem was conjectured to be coNP-hard in [4]. The problem remained 
open for nearly two decades. Recently, in a major breakthrough, this problem 
was shown to be coNP-complete by Vardy [18]. While this does not directly rule 


out the possibility that a good bounded distance decoding algorithm may exist, 
the result should be ruled as one more reason that general positive results may 
be unlikely. 

Thus we move from general results, i.e., where the code is specified as part 
of the input, to specific results, i.e., for well-known families of codes. The first 
question that may be asked is: “Is there a family of asymptotically-good [n, k, d], 
linear code and ¢ > 0, for which a polynomial time bounded distance decoding 
algorithm exists for ¢ > ed?” For this question the answer is “yes”. A large num- 
ber of algebraic codes do have such polynomial time bounded distance decoding 
algorithms. In particular the Reed Solomon codes are known to have such a 
decoding algorithm for t < |(d — 1)/2] (cf. [2, 11, 17]). This classical result is 
very surprising given the non-trivial nature of this task. This result is also very 
crucial for many of the known asymptotically good codes, since many of these 
codes are constructed by concatenating Reed Solomon codes with some other 
codes. In the next section we shall cover the decoding of Reed Solomon codes in 
more detail. 

Lastly there is another class of codes, constructed by combinatorial means, 
for which bounded distance decoding for some ¢ > ed can be performed in 
polynomial time. These are the expander codes, due to Sipser and Spielman [14] 
and Spielman [15]. The results culminate in a code with very strong — linear 
time (!!!) — encoding and bounded distance decoding algorithms. In addition 
to being provably fast, the algorithms for the encoding and decoding of these 
codes are surprisingly simple and clean. However, the description of the codes 
and analysis of the algorithm is somewhat out of the scope of this paper. We 
refer the reader to the original articles [14, 15] for details. 


5 Decoding of Reed Solomon code 


As mentioned earlier a polynomial time algorithm for bounded distance decoding 
is known and this algorithm corrects up to t < |(d — 1)/2] errors. Notice that 
this coincides exactly with the error-correction bound of the code (i.e., a Reed 
Solomon code of distance d is a t-error-correcting code for t = |(d—1)/2|). This 
bound on the correction capability is inherent, if one wishes to determine the 
codeword uniquely. However in the bounded distance decoding problem we do 
allow for multiple solutions. Given this latitude it is reasonable to hope for a 
polynomial-time decoding algorithm that corrects more errors - say up tot < 
(1—e)d where ¢€ is some fixed constant. However no such algorithm is known for all 
possible values of (n,k,d =n —k). Recently, in [16], we presented an algorithm 
which does correct up to (1 — €)d errors, provided k/n > 0. This algorithm 
was inspired by an algorithm of Welch and Berlekamp [20, 3] for decoding Reed 
Solomon codes. This algorithm is especially clean and elegant. Our solution uses 
similar ideas to correct even more errors and we present this next. 

Notice first that the decoding problem for Reed Solomon codes can be solved 
by solving the following cleanly stated problem: 


Problem 19 (Reed Solomon decoding). 

INPUT: n pairs of points {(#;,y%:)}, i, yi € GF(q); and integers t, k. 

Output: All polynomials p of degree at most & — 1 such that y, 4 p(x;) for at 
most ¢ values of 7. 


The basic solution idea in Welch-Berlekamp and our algorithm is to find 
an algebraic description of all the given points, and to then use the algebraic 
description to extract p. The algebraic description we settle for is an “algebraic 
curve in the plane”, i.e., a polynomial Q(x, y) in two variables « and y such 
that Q(a;,y:) = 0 for every value of « and y. Given this basic strategy, the 
performance of the algorithm depends on the choice of the degree of Q which 
allows for such a curve to exist, and still be useful! (For example if we allow Q to 
be 0, or if we pick the degree of @ be n in x and 0 in y, the such polynomials do 
exist, but are of no use. On the other hand a non-zero polynomial @ of degree 
n/10 in @ and 0 in y may be useful, but will probably not exist for the given 
data points.) 

To determine what kind of polynomial Q we should search for, we pick two pa- 
rameters / and m and impose the following conditions on Q(x, y) = er qty’: 


1. Q should not be the zero polynomial. (Le., some q;; should be non-zero.) 

2. qij is non-zero implies 7 < m and i+ (k —1)j < l. (The reason for this 
restriction will become clear shortly.) 

3. Q(xi, yi) = 0 for every given pair (x;, y;). 


Now consider the task of searching for such a Q. This amounts to finding 
values for the unknown coefficients q;;. On the other hand the conditions in 
(3) above amount to homogeneous linear equations in q;;. By elementary linear 
algebra a solution to such a system exists and can be found in polynomial time 
provided the number of equations (n) strictly exceeds the number of unknowns 
(i.e., the number of (7, 7) pairs such that 0 < i,j, 7 < mand i+(k—1)j < m). It 
is easy to count the number of such coefficients. The existence of such coefficients 
will determine our choice of m,/. Having determined such a polynomial we will 
apply the following useful lemma to show that p can be extracted from Q. 


Lemma 20 [1]. Let Q(z, y) = er qjr'y’ be such that qj; = 0 for every i, j 
with i+ (k —1)j > Ll. Then if p(x) is polynomial of degree k —1 such that for 
strictly more than l values of i, y = p(a:) and Q(#:,y;) = 0, then y — p(x) 
divides the polynomial Q(x, y). 


Proof. Consider first the polynomial g(«) obtained from Q by substituting y = 
p(x). Notice that the term q;;x'y’ becomes a polynomial in z of degree i+(k—1)j 
which by property (2) above becomes a polynomial of degree at most / in «. Thus 
g(x) = Q(«, p(x)) becomes a polynomial in x of degree at most /. Now, for every 
i such that y; = p(a;) and Q(a;, y;) = 0, we have that g(#;) = Q(a;, p(#;)) = 0. 
But there are more than / such values of 7. Thus g is identically zero. This 
immediately implies that Q(«, y) is divisible by y — p(x). (The division theorem 
for polynomials says that if a polynomial h(y) evaluates to 0 at y = ¢ then 


y — ¢ divides A(y). Applying this fact to the polynomial Q,(y) = Q(«, y) and 
y = p(x), we obtain the desired result. Notice in doing so, we are switching our 
perspective. We are thinking of Q as a polynomial in y with coefficients from 
the ring of polynomials in «.) 


Going back to the choice of m and I, we have several possible choices. In one 
extreme we can settle for m = 1 and then if 1 & (n+k)/2, then we find that the 
number of coefficients is more than n. In this case the polynomial Q(z, y) found 
by the algorithm is of the form A(«)y + B(x). Lemma 20 above guarantees that 
ift < |(n—k)/2| then y—p(a) divides Q. Thus p(x) = —B(x)/A(«) and can be 
computed easily by a simple polynomial division. Thus in this case we can decode 
from |(n — k)/2| errors thus recovering the results of [20]. In fact, in this case 
the algorithm essentially mimics the [20] algorithm, though the correspondence 
may not be immediately obvious. 

At a different extreme one may pick m & \/n/k andl & Vnk and in this case 
Lemma 20 works for t & n—2\/nk. In this case to recover p(x) from Q, one first 
factors the bivariate polynomial Q. This gives a list of all polynomial p;(x) such 
that y—p; (a) divides Q. From this list we pull out all the polynomials p; such that 
p;(:) # y; for at most t values of x;. Thus in this case also we have a polynomial 
time algorithm provided @ can be factored in polynomial time. Fortunately, such 
algorithms are known, due to Kaltofen [8] and Grigoriev [7] (see Kaltofen [9] for 
a survey of polynomial factorization algorithms). For k/n — 0, the number of 
errors corrected by this algorithm approaches (1 — o(1))n. 

A more detailed analysis of this algorithm and the number of errors corrected 
by it appear in [16]. The result shows that this given an [n,«n,(1—«)n], Reed 
Solomon code, the number of errors corrected by this algorithm approaches 


Lp 2 1 1 
1- —f8) where pg - |,f/o4—-—-]. 
n( + Dr ic) where p Ra 7 


A plot of this curve against « appears in Figure 1. Also shown in the figure 
are the distance of the code ((1 — «)n) and the classical-error correction bound 


(1 — &)/2n). 


6 Open questions 


Given that the fundamental maximum likelihood decoding problem is NP-hard 
for a general linear code, the next direction to look to is a bounded distance 
decoding algorithm for every [n,k,d], linear code. The bottleneck to such an 
approach is that in general we can’t compute d in polynomial time, due to the 
recent result of Vardy [18]. Thus the next step in this direction seems to suggest 
an application of approximation algorithms: 


Open Problem 1 Given ann xk matrir G, approrimate the distance d of the 
code Cg to within a factor of a(n). 
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Fig. 1. Fraction of errors corrected by the algorithm from [16] plotted against the rate 
of the code. Also plotted are the distance of the code and the classical error-correction 


bound. 


The goal here is to find the smallest factor a(n) for which a polynomial time 
approximation algorithm exists. Currently no non-trivial (i.e., with a(n) = o(n)) 
approximation algorithm is known. A non-trivial a(n) approximation algorithm 
would then suggest the following candidate for bounded distance decoding: 


Open Problem 2 Given an nx k matriz G, a word R € &” and an integer 
t, find all codewords within a Hamming distance of t from R, or show that the 
minimum distance of the code is less than tay(n). 


A similar problem is posed by Vardy [18] for a; = 2. Here the hope would 
be to find the smallest value of a, for which a polynomial time algorithm exists. 
While there is no immediate formal reasoning to believe so it seems reasonable 
to believe that a, will be larger than a. 

Next we move to the questions in the area of design of efficient codes, moti- 
vated by the work of Spielman [15]. 


Open Problem 3 For every k > 0, design a family of [n, Kn, dn]2 codes Cy, so 
that the bounded distance problem on C,, with parameter t < yn can be solved in 


linear time. 


The goal above is to make ¥ as large as possible for every fixed «. Spielman’s 
result allows for the construction codes which match the best known values of 6 
for any [n, Kn, dn]z linear code. However the value of ¥ is still far from 4 in these 
results. 

We now move towards questions directed towards decoding Reed-Solomon 
codes. We direct the reader’s attention to Figure 1. Clearly every point above 
the solid curve and below the distance bound of the code, represents an open 
problem. In particular we feel that the following version maybe solvable in poly- 
nomial time: 


Open Problem 4 Find a bounded distance decoding algorithm for an [n, Kn, (1- 
k)n]q Reed Solomon code that decodes up to t < (1 — V/k)n errors. 


The motivation for this particular version is that in order to solve the bounded 
distance decoding problem, one needs to ensure that the number of outputs (i.e., 
the number codewords within the given bound t) is polynomial in n. Such a 
bound does exist for the value of t as given above [6, 12], thus raising the hope 
that this problem may be solvable in polynomial time also. 

Similar questions may also be raised about decoding multivariate polyno- 
mials. In particular, we don’t have polynomial time algorithms matching the 
bounded distance decoding algorithm from [16], even for the case of bivariate 
polynomials. This we feel may be the most tractable problem here. 


Open Problem 5 Find a bounded distance decoding algorithm for the bivariate 
polynomial code Cpoty,2,nn,n that decodes up tot < (1- V2K)n? errors. 
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